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ABSTRACT 


The purpose of this paper is to present exten- 
sions of results announced by A. Ben- Israel concern- 
ing an iterative method for computing the generalized 
inverse of an arbitrary complex matrix. Ben- Israel 
announced his results without proof ; at about the 
same time, the authors of this paper independently 
derived very similar results with more relaxed hy- 
potheses. These similar results are presented with 
proof, together with comments pertaining to the 
Ben- Israel theorem. 
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AN ITERATIVE METHOD FOR COMPUTING THE 


GENERALIZED INVERSE OF A MATRIX 

By H. P. Decell, Jr. , and S. W. Kahng* 
Manned Spacecraft Center 


SUMMARY 


This paper presents extensions of results announced by A. Ben- Israel 
(ref. 1) concerning an iterative method for computing the generalized inverse 
of an arbitrary complex matrix. Ben-Israel announced his results without 
proof ; at about the same time, the authors of this paper independently derived 
very similar results with more relaxed hypotheses. These similar results 
are presented with proof, together with comments pertaining to the Ben-Israel 
theorem. 


INTRODUCTION 


A. Bjerhammar (ref. 2), E. H. Moore (ref. 3 ), and R. Penrose 
(ref. 4 ) independently generalized the concept of matrix inversion to include 
arbitrary complex matrices. The generalized inverse of a singular or non- 
square matrix possesses properties that make it a central concept in matrix 
theory as well as a very useful applied tool in statistical estimation, curve 
fitting, controllability of linear dynamical systems, stability theory, and so 
forth (refs. 5 through 14 and 16, 17, and 19). 

One of the equivalent definitions of the generalized inverse of an arbi- 
trary complex matrix is an immediate consequence of theorem I 
due to R. Penrose (ref. 4 ) stated here without proof. 


'* ' 
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Theorem I (Penrose) 


The four matrix equations 


AXA = A 

( 1 ) 

XAX = X 

( 2 ) 

(XA)* = XA 

( 3 ) 

(AX)* = AX 

( 4 ) 


have a unique solution X, for each complex matrix A. 

+ 

The unique solution X in theorem I is denoted X = A and is called 
the generalized inverse of A. In addition, it follows immediately from this 

4. 

definition that if A is square and nonsingular, then A is the usual inverse 
of A (that is, in classical notation, A + = A - *). 


SYMBOLS 


A a complex matrix 

A matrix conjugate transpose of A 

A" * matrix inverse of nonsingular A 

A + generalized inverse of A 

B a complex matrix 

b.. elements of the matrix B 

13 

k, n positive integers 
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r 


P R(B) 


R(B) 


R(B) X 

tr A 

X 

X 

n 

e 

x. 

i 


IX, £ 

£ 


orthogonal projection on range space of B 

range space of B 

orthogonal complement of R(B) 
trace of the matrix A 
unknown matrix 
matrix iterate 

zero matrix or vector as indicated by context 
eigenvalues 

vectors 
eigenvector 
matrix norm 


SOME BASIC CONSIDERATIONS 


In an effort to make this paper self-contained it is necessary to state 
some basic lemmas concerning generalized matrix inversion and fundamental 
matrix theory. Some lemmas will be stated without proof, but with ample 
reference. 


Lemma I 

The generalized inverse A + of A (as defined by theorem I) is the 
unique solution of the two matrix equations 


AX = 


P R(A) 


( 5 ) 
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( 6 ) 


■III 

** = Pr(x) = \(a*) 


where R(A), R(X), and r(a ) , respectively, denote the range space of A, 
* 

X, and A , and where P^^ , and P / *\> respectively, denote 

R \A / 

the orthogonal projection operators on R(A), R(X), and r(a ) . 


Proof. - From theorem I it follows that AX = AA + and XA = A + A are 
hermitian idempotent matrices and hence are projection operators on the 
desired range spaces. 


Lemma II 


For the generalized inverse 


A + of A (ref. 


4) 


4. * * 

AAA = A 


* + 

= A AA 


(?) 


(a + ) + = A (8) 

UT . ( A + )* (9) 


Definition I 

The norm of a square matrix B is a non-negative nu mb e r 
(denoted | | B | | ) which satisfies (ref. 15). 


| |B| | > 0, if B/0, | |e| | = 0 (10) 

| |cB| | = | c | ' | | B | |, for any complex number c (11) 
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B + D| I g | |B| | +| |D| | (12) 

D| | (13) 

Definition II 

A sequence of square matrices B (n = 0, 1, 2, . . .) is said to converge 
in norm | | | | to the matrix B ^written B^ 11 1 ! » . B^ provided that the 

sequence of real numbers | | B r - B | | converges to zero. 

Following (ref. 15), a number of basic norms are defined, and some 
comparison inequalities are stated. 

Lemma in 

If B is a k by k square matrix then the following equalities define 
norms satisfying definition I (ref. 15) 



B 


./max A 

V i 1 


(14) 


-T* 

where . . . , are the eigenvalues of B B (see lemma V). 


B = max 
i 


k 

y i b -. 
i=i 1 


(15) 


B| I 2 = max 


3 i=l 


k 


(16) 



B 


0 = k max 

b.. 




3 . . 

1,3 

U 


(17) 
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| b || 4 = p. s, | by | 2 = /trB*B (18) 


Lemma IV 

The norms in lemma III satisfy the following inequality (ref. 15) 


JsIMMeIM / ¥ || b | I < 19 > 


Lemma V 

* * 

For any matrix A, the eigenvalues of AA and A A are real, identi- 
cal, and non- negative. 


ITERATIVE COMPUTATION OF A + 

Ben- Israel initially published the following theorem (ref. 18) in an 
attempt to give a useful iterative scheme for computing A + . This theorem 
used the equivalent definition of A + given in lemma I. 

Theorem II (Ben- Israel) 

The sequence of matrices defined by 

X t = X (2P ry/A . - AX \ 
n+1 n \ R(A) n ) 

converges in any | | | | -norm defined by lemma III to the generalized inverse 
of A, provided 


X Q = A Bq, for some nonsingular Bq (20) 
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X 0 = C 0 A . 

for some nonsingular Cq 

1 I p r(a>- ax o| 

i<* 

IP . X n A 1 

i< ! 

1 r(a*) 0 1 


( 21 ) 

( 22 ) 

( 23 ) 


Note here that the term "the sequence X n converges in | | | | -norm to 
the generalized inverse of A" means that ^since X n may be a rectangular 
matrix for which a norm, in our sense, is not defined) 



( 24 ) 



( 25 ) 


Theorem II requires a good deal of hypotheses. Inparticular, it 
requires a priori knowledge of the projections P D / A v and P , . How- 

K ^ A ' r(a ) 

ever, as Ben- Israel notes, a few more direct computations will produce the 
generalized inverse without iteration in this particular case. 

During preparation for the publication of these results, Ben- Israel an- 
nounced without proof very similar results (ref. 1). In fact, the statement of 
the main result in theorem III will closely parallel the statement of the 
Ben- Israel results announced in reference 1. A corollary will follow 
theorem in in order to point out results apparently unnoticed by Ben- Israel. 
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Theorem III 


Let A be a q by p matrix (nonzero), be the largest eigenvalue 

of AA , and Xq = a A , where 0 < a < 1/X . The sequence defined by 


X 1 = X 
n+1 


n^'^n) ( n = 


= 0 , 1 , 2 , 



converges in any | | | | -norm defined in lemma ID to the generalized inverse 
of A. 


Again, note that this convergence is that convergence defined by equa- 
tions (24) and (25). 

Proof.- The following facts will be established in order to prove the 

* 

desired results for Xq = a A satisfying the hypothesis 



The indicated norm in equations (28) and (29) will be the square root of the 

* 

largest eigenvalue of B B defined in lemma in. Once these facts are estab- 
lished the proof will be complete since equations (26) through (29), together 
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with equation (13) of definition I and lemma IV, imply the convergence (in any 

norm of lemma III) of P_, A s - AX and P M . - X A to fl. 

kia; n r(a / n 


In order to establish equation (26) note, from the recursive computation 

of X , that there exist matrices F , (n = 0, 1, 2, . . . ) such that 
n’ n ' 


X = F A 
n n 


so that, using equations (1), (5), (7), and (30), 


AX P 0 / AN = AF A AA = AF A = AX 
n R(A) n n n 


P, aN AX = MAX = AX 
R(A) n n n 


From equations (31) and (32), observe that 


P D/A , - AX = P 0 / A x - 2 AX + AX AX 
R(A) n J R(A) n n n 


■ P R(A) - "S,? 1 - "SO 


- p - AX , 
R(A) n+1 


A dual argument will establish equation (27). 


I 


In order to establish equation (28), first note that if A is an eigenvalue 

)jc 

of AA then 1 - aA i is an eigenvalue of I - aAA . This fact will be of 

+ * 

importance in the examination of the eigenvalues of AA - a AA leading to 
the proof of equation (28). However, it will first be necessary to prove that 

+ * 

the nonzero eigenvalues of AA - aAA are of the form 1 - a A. where X^ 

is a nonzero eigenvalue of AA . To this end, let | ^ 0 be an eigenvector of 
AA + - aAA with the associated eigenvalue A^ . The vector i; / 6 can be 

written as the sum £ = p + r ] , where peR(A) and rjeR(A) ± (ref. 5) and hence 
it follows that 

(AA + - aAA*) ^p + rfj = A^p + r\ J (33) 

(aA + - aAA*AA + ) (p + r\] = (aA + - aAA ) (p + v) (34) 

so that by lemma I 

% 

p - aAA p = A^p + \^t] (35) 


Multiplying both sides of equation (35) by AA + (and using eqs. (1) and (5)) it 
follows that 





* 

aAA p 
aAA )p 


A|P 

A^P 


(36) 

(37) 
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Now if ji £ 9 , then equation (37) implies that X^ is also an eigenvalue 

of (i - aAA ) . In this case, as mentioned previously, X^ must be of the 
form 


X^ = 1 - aX t (38) 

for some eigenvalue of AA . Moreover, for the case |i ^ 9 , it will be 
shown that the X^ in equation (38) are different from zero. To this end, note 
that if X^ = 0 and ju ^ 9 then X^ = 1 so that equation (36) implies 

AA jit = 9 (39) 


* 

Multiplying both sides of equation (39) by A + A + it follows that 


A + A + AA*fX = A + A*fi = (AA + ) p = AA + \i = 9 (40) 


This is impossible since ju / 9 and /j.eR(A). Indeed, AA + is the orthogonal 
projection on the range of A so that 


AA + |i = \x ^ 9 (41) 

contrary to equation (40). 

Considering the case jj, = 9 , it follows from equation (35) that X^ = 0 . 

+ * 

Hence, the nonzero eigenvalues of AA - aAA are of the form 1 - aX 
where X^ is a nonzero eigenvalue of AA . Moreover, all of the eigenvalues 


11 



AA are non-negative so that for 0 < a < where | X m /0 is the 

largest eigenvalue of AA ) 


1 - aX. < 1 
i 


( 42 ) 


Since , as mentioned at the outset of the proof, 


*R(A) ~ | | = | | AA + - a AA | | is the square root of the largest 

eigenvalue of 


(aA + - aAA*) (aA + - aAA*) - (aA + - aAA*) 


(43) 


l + *\ 2 

it follows that the nonzero eigenvalues of ( AA - aAA j are of the form 


(l - aA.) 2 < 1 


(44) 


so that 



* 

aAA 



* 

where the A are nonzero eigenvalues of AA . Hence 


AA + - aAA I j < 1 


which is in fact equation (28). 
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/ ^ 
A dual argument will establish equation (29) (using the fact that AA 

and A A have the same eigenvalues) and, thus, complete the proof of the 
theorem. 

The following corollary will eliminate the need for computing the 
* 

eigenvalues of AA in theorem III. 


Corollary I 

In theorem HI, the choice of a may be limited to 0 < a < 1/ (3 

* 

where (3 is any norm defined in lemma III of AA . 

Proof - The eigenvalues of any square matrix B cannot exceed any of 

the norm of B defined in lemma III (ref. 15). Hence, if /3 is any norm 
* 

of AA defined by lemma in then 


0 < 


A 

m 


^ P 


so that 



1 

A 

m 


It follows that theorem HI is valid for any choice of a such that 
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CONCLUDING REMARKS 


The algorithm described in theorem III always guarantees the knowledge 

of the proper initial guess to force convergence. In addition, corollary I rules 

* 

out the necessity for calculating eigenvalues of the matrix AA in order to 
find a suitable constant a. This item did not appear in reference 1. The 
algorithm in theorem III was successfully tested on Hilbert segments through 
order seven. 


Manned Spacecraft Center 

National Aeronautics and Space Administration 
Houston, Texas, May 2, 1966 
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